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There is consensus within’ the — scientific 
community that replication studies are 
scientifically sound and bring added value and 
reliability to any discovery. It is also widely 
acknowledged that replications are not as 
common as they perhaps should be. The reasons 
for this vary depending on the stakeholder - here 
we will discuss the matter from the perspective of 
the pharmaceutical industry. However, the views 
expressed here are our own and do not represent 
the official standpoint of our employer or the 
industry in general. We _ will focus this 
commentary on the issues related to replications 
of the clinical part of the drug development, as 
our expertise is most closely related to this area, 
but the matter is equally relevant and important in 
the pre-clinical domain. 

The development of novel drugs is motivated by 
the pursuit of more effective, safer, and better 
tolerated treatment options. The process is 
formalized and regulated in order to minimize the 
possibility that any harm comes to any person 
using the drug and to maximize the possibility 
that only medications that have proof of a 
significant benefit over the standard of care are 
placed on the market. After the initial drug 
discovery and development, the preclinical stage 
aims to provide detailed information on dosing 
and toxicity levels using animal models or in 
vitro. These are followed by human studies, that 
are conducted in phases; phase 0/1 focuses on 
safety and dosage on 10-30 healthy human 
volunteers, phase 2 on safety and efficacy with a 
limited number of patients of the disease in 
question, whereas phase 3 studies are typically 
large, multicenter trials with hundreds or 
thousands of patients where the novel drug is 
compared to the standard of care and/or placebo. 
In many cases the control in phase 2 and 3 are 
analogous to a positive or active control, 
especially if the standard of care has been used for 
a long time before the study. The health 
authorities may also require that multiple phase 3 
trials are carried out, in order to validate the 
efficacy and safety profile through replication 
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before market authorization. Moreover, requests 
for post market authorization commitments to 
conduct phase 4 trials studies to further evaluate 
the efficacy and safety of a drug are becoming 
more common. 

As detailed above, the drug development process 
involves many studies and is typically both long 
and expensive. The incentive to base business 
decisions on reliable and replicable scientific data 
is therefore not only an ethical one, it is also 
financially important. In other words, it 1s worth 
investing money to acquire reliable replicable 
data, as this is not only a prerequisite for market 
authorization but will also minimize the financial 
risks involved in drug development. Replications 
of clinical human studies are important, as these 
increase the amount of data on safety and 
tolerability, confirm (or refute) a previously 
observed therapeutic effect and offer access to 
novel treatment options to patients. It is worth 
noting that, in a sense, the stages of the clinical 
drug development process is a type of replication 
setup (especially with regards to phases 2-4), 
where the following phase replicates the previous 
phase in a larger sample. 

As is currently the case in many, if not most 
scientific fields, pure replications of clinical 
studies where every parameter is the same are 
rare, though some of the reasons for this are 
unique to the clinical setting. While in theory 
there are no ethical objections to a replication 
study in humans, given that (as is standard) the 
study follows the declaration of Helsinki and is 
pre-registered, each study must always be 
carefully evaluated by an independent review 
board before it starts. For example, it may not be 
ethical to replicate a pure placebo-controlled trial 
if there is sufficient evidence of an efficient 
treatment for the disease. Another major obstacle 
is the resources needed to collect hundreds or 
even thousands of patients across the globe, 
making sure that the protocol is followed 
identically at every site. 

However, there are several ways to conduct 
studies that at least partially capture the objectives 


of experimental replications. For example, a 
clinical experiment that in addition to the 
replication, includes other research questions, 
such as a novel control group is usually a 
financially sound way to acquire replication data. 

Pharmaceutical companies also often commit to 
further test the efficacy and safety of their new 
drugs in long term follow-up studies when 
applying for marketing authorization or 
reimbursement. These real-world-evidence studies 
use observational data (real-world data, RWD) 
obtained in the routine clinical practice and are 
important to show that the effect demonstrated in 
a well-controlled experimental trial replicates in a 
real-world naturalistic setting. While real-world 
studies are a cost-effective way to collect data 
from a population that exceeds the original 
clinical study, there are limits to what data is 
available, as the data is observational in nature. 
For example, in psychiatry treatment outcomes 
are not measured from laboratory tests, and the 
questionnaires or the formalized clinical 
assessment used in the trials may not be 
commonly used in the clinical practice. Instead 
treatment effects may need to be approximated 
from other variables, such as sick leave. In 
addition to the replication value of RWD, it can 
also provide new data that is not possible to get 
from a randomized clinical trial. For example, 
comorbidities are typically an exclusion criterion 
in clinical trials, yet it is important that the 
efficacy and safety of a drug is comparable also in 
patients with comorbidities. Also, adherence to 
the treatment is often different between clinical 
trials with frequent follow-up’ visits and 
real-world practice. Nonetheless, evidence 
generation through RWD is becoming an 
increasingly important part of value-based 
healthcare and provides an important way to 
replicate data. 

Another form of replication of clinical data takes 
place through independent investigator studies. 
The pharmaceutical company is not involved in 
these studies but may in some cases provide 
funding or sponsor the drug being investigated. In 
contrast to phase 3 clinical trials, these are 
typically performed in one or a few research 
centers, and with a substantially smaller study 
population. Also, these studies tend to have other 
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research objectives that go beyond a _ pure 
replication of a previous clinical study, as a 
replication with a smaller sample size than the 
original study defeats the purpose of a replication. 
Their goal is often to close remaining data gaps or 
to serve as ‘proof-of concept’ studies that instigate 
extensive clinical testing, instead of serving as 
replication-studies themselves. 

In the discussion of the value and importance of 
replication studies, it is sometimes forgotten to 
highlight that the interpretation of how a 
replication study impacts the findings of the 
original study is not always straightforward. If the 
original study and the (partial)replication vary on 
one or more factors, caution is warranted in the 
interpretation (though it does not render the 
replication useless by any means). Some 
examples of factors that may make _ the 
comparison of two studies very difficult are 
differences in sample size, effect size, choice of 
statistical method, duration of the — study, 
background demographic factors or the route and 
dose of medication. To be able to compare studies 
with each other, replications studies would benefit 
from more formalized standards similar to those 
that are applied in systematic reviews and 
meta-analyses. 

In conclusion, we believe that all stakeholders 
should strive to remove the barriers that currently 
make them reluctant to repeat studies. Some 
barriers are more easily addressed than others. 
Understanding the importance of replication 
studies makes science better, and at the end of the 
day it is the patients that will benefit the most 
from replication studies, be they in the domain of 
preclinical compound development or extensive 
clinical studies. Inspired by the 3R rule for 
preclinical animal studies (replace, reduce, 
refine), we advocate for a similar 3R rule for 
replication studies, that is even easier to learn: 
repeat, repeat and repeat! 
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